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Abstract 

Science students who perform poorly in introductory science classes often don t know how to do better 
because they "don t know that they don t know” and they have poor critical thinking and problem solving 
skills. The current study describes how modeling and coaching students in "active study” improves higher 
order thinking skills. The data show that students who received such coaching performed significantly better 
on exams by the end of the semester compared to control groups. Modeling and coaching "active study” 
behavior is essential to ensure that students build confidence in their ability to study science and remain in the 
science pipeline. 
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Abstract 

Science students who perform poorly in introductory science classes often don't know 
how to do better because they "don't know that they don't know", and they have poor 
critical thinking and problem solving skills. The current study describes how modeling 
and coaching students in "active study" improves higher order thinking skills. The data 
show that students who received such coaching performed significantly better on exams 
by the end of the semester compared to control groups. Modeling and coaching "active 
study" behavior is essential to ensure that students build confidence in their ability to 
study science and remain in the science pipeline. 


Introduction 

Introductory college courses are a first step in the process of a student's development of 
critical, reading, thinking, and writing skills. More than just mastering basic content 
knowledge of a specific area of study, one goal of introductory courses is for students to 
"learn", or understand and use course content to demonstrate higher order thinking 
skills like, analysis, synthesis, and evaluation of problems (according to Bloom's 
taxonomy of critical thinking skills, Bloom 1964). This problem is more difficult in the 
sciences for a variety of reasons. There is an increasingly complex volume of "content" 
(i.e. facts) to be mastered, and lecture periods often proceed at a much more rapid pace 
than high school science classes on similar subjects. Although college instructors may 
use a variety of active learning techniques in their introductory science courses, there 
may still be a significant portion of class time devoted to lecture. First year college 
students often have poor listening, note taking, and synthesizing skills (Erickson and 
Strommer 1991) which hinders their learning in the classroom. Poorly developed study 
skills combined with poor time management and poor self-discipline in first year college 
students may result in them feeling so lost and overwhelmed that they give up easily on 
courses they otherwise profess great interest in and should be able to master. This is 
the point at which instructors need to intervene and provide support and direction for 
helping students "learn how to learn." 

There are a wealth of science education resources that provide strategies for encouraging 
interaction in the classroom and promoting deeper levels of intellectual engagement of 
students (e.g., Angelo and Cross, 1993). Many of these pedagogical strategies address 
the problem of the poorly performing student that needs to adopt new study skills that 
improve critical thinking, that stress understanding rather than rote memorization, that 
help students move beyond the level of simple recognition of vocabulary that they 
mastered in high school to using that vocabulary to solve 

problems. Multiple publications, with suggestions specifically for improving critical 
thinking skills of students, abound in the literature (a few recent examples: Lauer 2005; 
Yuretich 2003/2004; Svinicki 2004; but see also texts on the subject by Stice 1987 and 
Halpern 2003). For example, Halpern (1998) proposed a four-part process involving: 
an attitudinal component, instruction and practice in critical thinking skills, activities that 
enhance transfer of these skills across a variety of contexts, and development of 
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metacognitive processes to monitor progress. Similarly, Svinicki (2004) has developed a 
survey instrument (GAMES©) in which students are introduced to different techniques for 
good study behavior; in the process of evaluating what they do when they study, they 
learn other, perhaps more efficient ways of learning and monitoring their success. 
Broadbear (2003) suggests that "ill-structured problems" (those with more than one 
right answer, arrived at through sound reasoning), some criteria for assessing critical 
thinking, (especially that done by the students themselves), and demonstration of 
improvement of thinking (e.g., portfolios) are the key elements to improving critical 
thinking ability. 

All of these methods have a common strategy, to engage students in metacognitive 
effort, that is, to improve self awareness of their learning, and similar outcomes, that of 
increasing students' reflection on and engagement with the subject material. Despite 
the quantity of publications describing the methods for improvement of critical thinking 
skills and exam performance in college students, the quantitative assessment of the 
efficacy of the methodology lags behind. The purpose of this report is to quantify the 
impact of a particular instructional intervention (modeling and coaching "active" 
studying) on a group of "at-risk" students in an introductory level biology course for 
biology majors. 


Methods 

In order to learn something about the level of awareness that students of varying 
academic ability have about their own learning, students enrolled in a large introductory 
biology course at University of St. Thomas, St. Paul, Minnesota (88 % freshmen, n=90 
students) were asked to predict their performance on every exam immediately after 
turning it in. Their actual scores were then compared with their predictions for each of 
four exams during the semester in order to determine longitudinal trends in their 
performance and their ability to predict their performance. 

To investigate techniques that might help "at-risk" introductory students reform their 
study strategies to improve both their actual performance and their ability to predict 
performance, students in the same introductory course (n=126) during a subsequent 
year were grouped voluntarily according to the type of interaction they had with the 
instructor: 1) 15 students who sought help from the instructor on several occasions 
during the semester (coached group), 2) 12 students who interacted with the same 
instructor during the laboratory portion of the course (lab contact group), and 3) 15 
students who did not seek help from the instructor during the semester (no contact 
group). Students in the coached group scored between 50% and 80% on the first exam 
and voluntarily sought help to improve their exam performance. The third group of no 
contact students was selected by matching their first exam scores (+ 2%) with those of 
the students in the coached group, thus making it possible to compare improvement in 
performance as a function of the coaching technique used by the instructor. Students in 
the lab contact group of 12 students not only interacted with the instructor on a more 
personal level in the small laboratory section weekly, but were also the instructor's 
freshman advisees. These students were concurrently enrolled in an advanced English 
literature course that was paired with the freshman biology lab, and thus tended to be 
better than average students. 

Students in the coached group received assistance from the instructor in a number of 
ways: exam analysis, demonstration of active study techniques, and Q and A on sample 
upcoming exam questions. Exam analysis consisted of looking for patterns of missed 
questions, misread questions and/or answers, voids in content knowledge, and 
misconceptions or incorrect notes. Students were asked to group the exam questions 
into easy, medium, and hard difficulty levels for them, and then assign one of three 
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critical thinking categories (knowledge, application, analysis) to the questions to 
determine which categories of questions they were answering incorrectly. Modeling and 
coaching methods of "active studying" provided students with several new techniques for 
learning. For example, one "active" studying approach modeled was building concept 
maps using the same flash cards students had created to study vocabulary. Students 
were asked to state why they drew the concept maps they way they did and to describe 
the nature of the relationship between the items. Another technique was to have 
students write an explanatory paragraph about a figure from the text that utilized new 
vocabulary or flash-card concepts, or conversely, to draw a figure based on a descriptive 
paragraph of bold-faced words from the text. Students in the "coached" group asked to 
be pre-tested on upcoming exam material by providing them with practice test 
questions. However, the instructor felt it was more beneficial for the students to write 
and critique test questions of their own construction. Students were urged to write and 
share questions with each other, occasionally in groups, and usually with the instructor 
present for guidance. Further, they were encouraged to write questions that illustrated 
knowledge-, application-, or analysis-type levels of thinking, as practice for what they 
might see on the actual exam. 


Results 

Low achievers on individual exams (scoring less than 60% on an exam) consistently 
overestimated their performance by an average of 22%, while high achievers (scoring at 
least 90% on the exam) were either very accurate or slightly underestimated by 5% 
their actual performance. Although students improved slightly in their ability to predict 
their exam performance over the course of the semester, students who performed poorly 
still over-predicted their performance by an average of 17% on the third exam (Fig. 1). 



Figure 1 . Longitudinal (over the semester) changes in the mean percent difference between 
predicted and actual exam score for 90 introductory biology students scoring in specific grade 
categories A (90-100%), B (80-89%), C (70-79%), D (60-69%) and F (<60%) on each exam. 
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There was a surprisingly significant inverse linear correlation between the mean 
difference between predicted and actual exam score and the mean score of the grade 
category on an exam: 

% difference = - 0.66* (mean score of grade category) + 56.20; 

R 2 = 0.98; P = 0.001. 

There was a significant and positive impact of the modeling and coaching strategy on 
exam performance of students in the coached group (Table 1). There was no significant 
difference between coached and no-contact groups on their first exam, as expected, 
because these students were matched for first exam performance. Both coached and 
no-contact groups scored significantly lower than the lab-contact group on the first 
exam, most likely because the lab-contact group was composed of strong students 
enrolled in this particular lab section because it was paired with an advanced Freshman 
literature course. However, there was a highly significant difference between the 
coached and no-contact groups on the last exam. Further, there was no significant 
difference between the coached group and the lab contact group on the last exam (Table 
1), as the coached group had increased their mean exam score almost 21 points from 
first to last exam of the semester. In contrast, the no-contact and lab-contact groups 
showed significantly less change from first to last exam scores (Table 1). 


Table 1 . Comparison of means and standard errors (in parentheses) of first and last (final) exam 
scores and difference between last and first exam scores of introductory biology students. All 
exams were 100 points. See methods for definition of groups. Students in the coached and no¬ 
contact groups were matched for first exam performance. Groups were compared by one-way 
ANOVA with post-hoc comparison of means by the Tukey test. Significant differences between 
means within the ANOVA are indicated by different alphabetic superscripts following the means. 


PARAMETER 

COACHED 

NO CONTACT 

LAB CONTACT 

ANOVA 

P 

Mean score first 
exam (S.E.) 

66.1 A (2.41) 

67.5 A (2.55) 

82.6 B (2.71) 

<0.0001 

Mean score last 
exam (S.E.) 

87.0 A (2.22) 

73.9 B (2.33) 

88.1 A (1.84) 

<0.0001 

Difference in 
exam scores, last 
to first (S.E.) 

+20.9 A (2.4) 

+6.4 B (2.1) 

+ 5.5 B (2.2) 

<0.0001 

Number of 
students 

15 

15 

12 



Coached students were surprised at how much the exams revealed about the gaps in 
their study preparation and about how much carelessness and inaccuracy had 
contributed to wrong answers. Further, when they were able to (correctly) identify the 
levels of critical thinking involved in each question, they could evaluate the types of 
questions that were more frequently missed on the exam and modify their method of 
study preparation accordingly. This type of exam analysis seemed to be as beneficial in 
stimulating a change in study habits as the coaching of "active studying". 

Asking students to write (and share with each other) their own questions that illustrated 
the knowledge, application, or analysis level of thinking proved far more helpful as a 
study aid than the flash card memorization technique, and it encouraged development of 
their critical thinking skills. In reviewing the construction of student-written exam 
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questions with the instructor, students began to adopt a more metacognitive approach to 
their learning, assessing what they did or did not know in order to construct a good 
question. By writing sample test questions themselves students were forced to focus on 
the central content of a particular unit and to learn to distinguish core ideas to be 
mastered from the tangential examples or illustrations. 


Discussion 

The strong negative correlation (Fig. 1) between the magnitude of error in predicting 
exam score and the actual exam performance indicates that low achieving students not 
only lack the content knowledge or critical thinking skills required to answer exam 
questions correctly, but they also do not recognize whether the answer they give may, in 
fact, be wrong. That is, students who perform poorly don't know that they don't know, 
and students who perform poorly don't know how to change their method of studying. 
This is not a new finding in the education or cognitive psychology literature (Kruger and 
Dunning 1999; Kennedy et al. 2002). However, what was unusual about these data was 
the goodness of fit of the data to a linear relationship between predictive error and exam 
performance (R 2 = 0.98). 

Although there was some improvement in the degree of predictive accuracy about exam 
performance of all groups of achievers (Fig. 1) over the course of the semester, this 
could simply be explained by student realization that they never performed as well as 
they thought, and should therefore predict a lower score for themselves on each 
successive exam. Thus, this is not good evidence that students get better at realistically 
assessing their level of learning throughout the semester. 

Low achievers typically complain that "I studied harder for this test than any other, and 
still got an F"; or "The problems were completely different than the homework"; or "The 
questions were tricky"; or "You never talked about this in class". They label questions 
as "tricky" or assume it was "not covered" in class because they are not able to apply or 
analyze content knowledge to solve problems that require a higher level of critical 
thinking than simple recall and recognition type questions they have mastered in the 
past. The challenge is to provide these students with a different approach to studying 
"hard". 

In this study, there was a marked (and significant) increase in exam performance of 
coached students who engaged in "active" studying tutorials, compared to the other two 
groups (Table 1). One might argue that students who seek help and have more personal 
contact with the instructor in a large class are bound to do better on exams, and that it 
is not the active studying method that is making a difference. However, in this study, 
the instructor spent as much or more informal time with the lab-contact group of 
students than the coached students. Thus, personal contact alone with an instructor is 
not enough to get students engaged in their own learning process. 

Positive effects of critical thinking exercises on exam performance have been reported by 
others, as well. Osborne (2000) reported that students who enrolled in a critical thinking 
laboratory received higher grades in their introductory psychology course, received fewer 
D, F, or withdrawal grades, and had higher overall GPA than students who enrolled in the 
introductory psychology course alone. Similarly, Yuretich (2003/2004) measured 
progressive and significant improvement in performance of geology students on exams, 
following implementation of critical thinking exercises in the classroom. 


Educational Implications 
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For students whose exam preparation formerly consisted of reading notes many 
times and making flash cards of definitions of concepts or vocabulary, "active studying" 
is both foreign and laborious. Modeling the process of "active studying" alone is not 
sufficient because introductory students are novices with the use of most pedagogical 
techniques and may, in fact, use them incorrectly, thus contributing to their frustration 
when these "new" techniques fail them. For example, the concept map technique 
requires not only modeling examples for the student, but careful critique of the content 
and accuracy of the map. As Bransford et al. (1999) point out, novices, such as 
introductory students, may build maps with missing (essential) content and with 
incorrect relationships, and in studying this, may gain erroneous misconceptions. The 
concept map technique does not work well unless the instructor goes over the map with 
the student, listening carefully to their explanations and correcting their misconceptions 
(coaching). Similarly, writing sample test questions encourages deeper learning and 
thoughtful analysis in students who might otherwise have resorted to simply memorizing 
terms. However, instructor guidance is essential in this task to ensure that students' 
questions (and answers) stretch beyond the simple knowledge recall level of thinking. 

What can instructors do to reach struggling students who don't voluntarily seek help 
from the instructor? Coached students made the greatest gains in metacognitive 
processing by reviewing the reasons for missed questions on exams and by analyzing 
the cognitive level of the test questions themselves. This is an activity from which all 
students can profit and takes little time for the instructor to demonstrate after the exam, 
or even after every exam. Although it seems obvious to instructors that gaps in 
knowledge may lead to lower exam scores, it is not apparent to students how much of 
an impact that knowledge gap actually has on their test score. When students realize 
the number of missed questions that result from one of these gaps, they may make 
more of an effort to fill in gaps in their notes or their understanding of the material 
before the exam. Another exercise that may help all students to improve exam scores is 
having them assign a Bloom category to the exam questions: recognition-recall, 
comprehension, application, analysis, etc., and then try to write questions like those for 
study on the next exam. There is often a mismatch between the types of questions that 
students expect to see on an exam or may write when they study (recognition-recall 
type questions) and the actual exam questions themselves that may require higher order 
processing than simple recognition. The more practice students have with writing and 
answering questions that require higher order thinking, the better they will do on exams. 

Modeling and coaching learning in the classroom, and then reviewing the students' 
progress are all essential to changing study habits and encouraging new methods of 
learning. For "coached" students in this study, the rewards were not only a marked 
increase in exam score, especially in the lower achieving students, but an increased self- 
confidence in their ability to "do" science. This is especially important in introductory 
science courses, from which there is typically a high attrition rate, due to large class 
size, and a lower level of student-instructor interaction and greater fear of failure than 
some humanities courses. Although there is no guarantee that we can reach these 
students before they decide to give up on a science career, taking proactive steps to 
ensure that introductory science students "learn how to learn" seems to be imperative 
for a continued flow of talent into the science pipeline. 
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